Seam processing for panorama weaving

ABSTRACT

An image mosaic is created from overlapping images. A first seam is shown between a first pair of overlapping images such that, on a first side, a first image of the first pair is shown, and on a second side, a second image of the first pair is shown. The first seam includes a plurality of pixels in the image mosaic. An indicator of an interaction with the image mosaic is received. The indicator indicates that the user has selected a pixel of the plurality of pixels of the first seam and has moved the selected pixel to a first location within an overlapping region between the first pair of overlapping images. A second seam that includes the first location is computed. The image mosaic including the second seam is presented. The second seam replaces at least a portion of the first seam in the image mosaic.

BACKGROUND

The composition of panoramas from a collection of smaller individualimages has recently become a popular application in digital imageprocessing. With the introduction of low-cost, robotic tripod headsalong with the improvement of image registration techniques, panoramasare becoming larger and more complex. In the past, these imagecollections were captured in one sweeping motion such that imageoverlaps were in only one dimension. Today's images are oftencollections of multiple rows and columns, and thus, overlap in twodimensions. Today's images further may be arranged in unstructuredconfigurations. The fully composited panoramic image can range from afew megapixels to many gigapixels in size. As a result, moresophisticated panorama processing techniques continue to be developed toaccount for their more complex configurations and larger sizes.

A fundamental step in stitching several pictures to form a larger mosaicis the computation of boundary seams that minimize the visual artifactsdue to the transition between images. Current seam computationalgorithms use optimization methods that may be slow, sequential, memoryintensive, and prone to finding suboptimal solutions related to localminima of the chosen energy function. Currently, the most used techniquefor global seam computation in a panorama is the Graph Cuts algorithm.For reference, see for example, R. Zabih et al., Fast Approximate EnergyMinimization Via Graph Cuts, Int'l Conference Computer Vision 377-384(1999); Y. Boykov & V. Kolmogorov, An Experimental Comparison ofMin-cut/Max-Flow Algorithms for Energy Minimization in Vision, 26 IEEETransactions on Pattern Analysis and Mach. Intelligence 9, 1124-1137(2004); and V. Kolmogorov & R. Zabih, What Energy Functions can beMinimized Via Graph Cuts?, 26 IEEE Transactions on Pattern Analysis andMach. Intelligence 2, 147-159 (2004). This is a popular and robustcomputer vision technique and has been adapted to compute the boundarybetween a collection of images. While this technique has been used withgood success for a variety of panoramic or similar graphicsapplications, it can be problematic due to its high computational costand memory requirements. Moreover, Graph Cuts applied to digitalpanoramas is a serial operation. Additionally, even when the variousknown techniques perform well, what they compute may not provideperceptually ideal or even good seams between the input images.

SUMMARY

In an example embodiment, a method of allowing a user to modify an imagemosaic is provided. The image mosaic is presented in a display of afirst device under control of a processor. The image mosaic is createdfrom a plurality of overlapping images. A first seam is shown between afirst pair of the plurality of overlapping images. On a first side ofthe first seam a first image of the first pair is shown and on a secondside of the first seam opposite the first side a second image of thefirst pair is shown. The first seam includes a plurality of pixels inthe image mosaic. An indicator of an interaction with the image mosaicis received. The indicator indicates that the user has selected a pixelof the plurality of pixels of the first seam and has moved the selectedpixel to a first location within an overlapping region between the firstpair of overlapping images. A second seam that includes the firstlocation is computed. A second seam that includes the new location iscomputed. The image mosaic including the second seam between the firstpair of the plurality of overlapping images is presented in the displayof the first device under control of the processor. The second seamreplaces at least a portion of the first seam in the image mosaic.

In another example embodiment, a computer-readable medium is providedhaving stored thereon computer-readable instructions that when executedby a computing device, cause the computing device to perform the methodof allowing a user to modify an image mosaic.

In yet another example embodiment, a system is provided. The systemincludes, but is not limited to, a display, a processor and acomputer-readable medium operably coupled to the processor. Thecomputer-readable medium has instructions stored thereon that whenexecuted by the processor, cause the system to perform the method ofallowing a user to modify an image mosaic.

Other principal features and advantages of the invention will becomeapparent to those skilled in the art upon review of the followingdrawings, the detailed description, and the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative embodiments of the invention will hereafter be describedwith reference to the accompanying drawings, wherein like numeralsdenote like elements.

FIG. 1 depicts a block diagram of an image creation and processingsystem in accordance with an illustrative embodiment.

FIG. 2 depicts a block diagram of an image processing device of theimage creation and processing system of FIG. 1 in accordance with anillustrative embodiment.

FIG. 3 depicts a block diagram of a data collection device of the imagecreation and processing system of FIG. 1 in accordance with anillustrative embodiment.

FIG. 4 depicts a block diagram of a data storage system of the imagecreation and processing system of FIG. 1 in accordance with anillustrative embodiment.

FIG. 5 depicts a block diagram illustrating interactions among thecomponents of the image creation and processing system of FIG. 1 inaccordance with an illustrative embodiment.

FIG. 6 illustrates use of an image processing application of the imageprocessing computing device of FIG. 2 in accordance with an illustrativeembodiment.

FIG. 7 depicts a flow diagram illustrating example operations performedby the image processing application in accordance with an illustrativeembodiment.

FIGS. 8 a and 8 b illustrate a seam between a pair of images inaccordance with an illustrative embodiment.

FIGS. 9 a and 9 b illustrate a min-cut solution and a min-path solutionfor a seam between a pair of images in accordance with an illustrativeembodiment.

FIG. 10 illustrates two min-path trees in accordance with anillustrative embodiment.

FIGS. 11-14 illustrate a process for determining a branching point inaccordance with an illustrative embodiment.

FIGS. 15-17 illustrate user interface windows presented under control ofthe image processing application of the image processing computingdevice of FIG. 2 in accordance with an illustrative embodiment.

DETAILED DESCRIPTION

With reference to FIG. 1, a block diagram of an image creation andprocessing system 100 is shown in accordance with an illustrativeembodiment. In an illustrative embodiment, image creation and processingsystem 100 may include a data storage system 102, an image processingsystem 104, a data collection system 106, and a network 108. Datacollection system 106 collects data associated with one or more images.Image processing system 104 processes the collected data associated withthe one or more images and creates an image mosaic from the collecteddata that includes seams that stitch the smaller individual imagestogether. Data storage system 102 stores data associated with the one ormore images and the image mosaic. An image mosaic, as used herein, is acollection of smaller individual images that are combined to form asingle image. In an illustrative embodiment, at least three of the oneor more images overlap at one or more pixels in the image mosaic.

The components of image creation and processing system 100 may beincluded in a single computing device, may be positioned in a singleroom or adjacent rooms, in a single facility, and/or may be remote fromone another. Network 108 may include one or more networks of the same ordifferent types. Network 108 can be any type of wired and/or wirelesspublic or private network including a cellular network, a local areanetwork, a wide area network such as the Internet, etc. Network 108further may be comprised of sub-networks and consist of any number ofdevices.

Data storage system 102 may include one or more computing devices. Theone or more computing devices send and receive signals through network108 to/from another of the one or more computing devices of data storagesystem 102, to/from image processing system 104, and/or to/from datacollection system 106. Data storage system 102 can include any numberand type of computing devices that may be organized into subnets. Datastorage system 102 may communicate with other computing devices usingvarious transmission media that may be wired or wireless as understoodby those skilled in the art. Data storage system 102 may communicateinformation as a peer in a peer-to-peer network using network 108.

Image processing system 104 may include one or more computing devices.The one or more computing devices of image processing system 104 sendand receive signals through network 108 to/from another of the one ormore computing devices of image processing system 104, to/from datastorage system 102, and/or to/from data collection system 106. Imageprocessing system 104 can include any number and type of computingdevices that may be organized into subnets. The one or more computingdevices of image processing system 104 may include computers of any formfactor such as a laptop 110, a desktop 112, a smart phone 114, apersonal digital assistant, an integrated messaging device, a tabletcomputer, etc. Image processing system 104 may include additional typesof devices. The one or more computing devices of image processing system104 may communicate using various transmission media that may be wiredor wireless as understood by those skilled in the art. The one or morecomputing devices of image processing system 104 further may communicateinformation as peers in a peer-to-peer network using network 108.

Data collection system 106 may include one or more computing devices.The one or more computing devices of data collection system 106 send andreceive signals through network 108 to/from another of the one or morecomputing devices of data collection system 106, to/from data storagesystem 102, and/or to/from image processing system 104. Data collectionsystem 106 can include any number and type of computing devices that maybe organized into subnets. The one or more computing devices of datacollection system 106 may include computers of any form factor such as alaptop 116, a desktop 118, a smart phone 120, an integrated messagingdevice, a personal digital assistant, a tablet computer, etc. Datacollection system 106 may include additional types of devices. The oneor more computing devices of data collection system 106 may communicateusing various transmission media that may be wired or wireless asunderstood by those skilled in the art. The one or more computingdevices of data collection system 106 further may communicateinformation as peers in a peer-to-peer network using network 108.

With reference to FIG. 2, a block diagram of an image processing device200 of image processing system 104 is shown in accordance with anillustrative embodiment. Image processing device 200 is an examplecomputing device of image processing system 104. Image processing device200 may include an input interface 204, an output interface 206, acommunication interface 208, a computer-readable medium 210, a processor212, a keyboard 214, a mouse 216, a display 218, a speaker 220, aprinter 212, and an image processing application 224. Fewer, different,and additional components may be incorporated into image processingdevice 200.

Input interface 204 provides an interface for receiving information fromthe user for entry into image processing device 200 as understood bythose skilled in the art. Input interface 204 may interface with variousinput technologies including, but not limited to, keyboard 214, display218, mouse 216, a track ball, a keypad, one or more buttons, etc. toallow the user to enter information into image processing device 200 orto make selections presented in a user interface displayed on display218. The same interface may support both input interface 204 and outputinterface 206. For example, a display comprising a touch screen bothallows user input and presents output to the user. Image processingdevice 200 may have one or more input interfaces that use the same or adifferent input interface technology. Keyboard 214, display 218, mouse216, etc. further may be accessible by image processing device 200through communication interface 208.

Output interface 206 provides an interface for outputting informationfor review by a user of image processing device 200. For example, outputinterface 206 may interface with various output technologies including,but not limited to, display 218, speaker 220, printer 222, etc. Display218 may be a thin film transistor display, a light emitting diodedisplay, a liquid crystal display, or any of a variety of differentdisplays understood by those skilled in the art. Speaker 220 may be anyof a variety of speakers as understood by those skilled in the art.Printer 222 may be any of a variety of printers as understood by thoseskilled in the art. Image processing device 200 may have one or moreoutput interfaces that use the same or a different interface technology.Display 218, speaker 220, printer 222, etc. further may be accessible byimage processing device 200 through communication interface 208.

Communication interface 208 provides an interface for receiving andtransmitting data between devices using various protocols, transmissiontechnologies, and media as understood by those skilled in the art.Communication interface 208 may support communication using varioustransmission media that may be wired or wireless. Image processingdevice 200 may have one or more communication interfaces that use thesame or a different communication interface technology. Data andmessages may be transferred between image processing system 104 and datastorage system 102 and/or data collection system 106 using communicationinterface 208.

Computer-readable medium 210 is an electronic holding place or storagefor information so that the information can be accessed by processor 212as understood by those skilled in the art. Computer-readable medium 210can include, but is not limited to, any type of random access memory(RAM), any type of read only memory (ROM), any type of flash memory,etc. such as magnetic storage devices (e.g., hard disk, floppy disk,magnetic strips, . . . ), optical disks (e.g., CD, DVD, . . . ), smartcards, flash memory devices, etc. Image processing device 200 may haveone or more computer-readable media that use the same or a differentmemory media technology. Image processing device 200 also may have oneor more drives that support the loading of a memory media such as a CDor DVD.

Processor 212 executes instructions as understood by those skilled inthe art. The instructions may be carried out by a special purposecomputer, logic circuits, or hardware circuits. Thus, processor 212 maybe implemented in hardware, firmware, or any combination of thesemethods and/or in combination with software. The term “execution” is theprocess of running an application or the carrying out of the operationcalled for by an instruction. The instructions may be written using oneor more programming language, scripting language, assembly language,etc. Processor 212 executes an instruction, meaning that itperforms/controls the operations called for by that instruction.Processor 212 operably couples with input interface 204, with outputinterface 206, with computer-readable medium 210, and with communicationinterface 208 to receive, to send, and to process information. Processor212 may retrieve a set of instructions from a permanent memory deviceand copy the instructions in an executable form to a temporary memorydevice that is generally some form of RAM. Image processing device 200may include a plurality of processors that use the same or a differentprocessing technology.

Image processing application 224 performs operations associated withcreating an image mosaic from one or more images and with editing thecreated image mosaic based on user interaction with image processingapplication 224. Some or all of the operations described herein may beembodied in image processing application 224. The operations may beimplemented using hardware, firmware, software, or any combination ofthese methods. With reference to the example embodiment of FIG. 2, imageprocessing application 224 is implemented in software (comprised ofcomputer-readable and/or computer-executable instructions) stored incomputer-readable medium 210 and accessible by processor 212 forexecution of the instructions that embody the operations of imagecreation and processing application 224. Image processing application224 may be written using one or more programming languages, assemblylanguages, scripting languages, etc.

Image processing application 224 may be implemented as a Webapplication. For example, image processing application 224 may beconfigured to receive hypertext transport protocol (HTTP) responses fromother computing devices such as those associated with data collectionsystem 106 and/or data storage system 102 and to send HTTP requests. TheHTTP responses may include web pages such as hypertext markup language(HTML) documents and linked objects generated in response to the HTTPrequests. Each web page may be identified by a uniform resource locator(URL) that includes the location or address of the computing device thatcontains the resource to be accessed in addition to the location of theresource on that computing device. The type of file or resource dependson the Internet application protocol. The file accessed may be a simpletext file, an image file, an audio file, a video file, an executable, acommon gateway interface application, a Java applet, or any other typeof file supported by HTTP.

With reference to FIG. 3, a block diagram of a data collection device300 of data collection system 106 is shown in accordance with an exampleembodiment. Data collection device 300 is an example computing device ofdata collection system 106. Data collection device 300 may include asecond input interface 304, a second output interface 306, a secondcommunication interface 308, a second computer-readable medium 310, asecond processor 312, a second keyboard 314, a second mouse 316, acamera 318, a second display 320, a second speaker 322, a second printer324, and a data collection application 326. Fewer, different, andadditional components may be incorporated into data collection device300.

Second input interface 304 provides the same or similar functionality asthat described with reference to input interface 204 of image processingdevice 200. Second output interface 306 provides the same or similarfunctionality as that described with reference to output interface 206of image processing device 200. Second communication interface 308provides the same or similar functionality as that described withreference to communication interface 208 of image processing device 200.Second computer-readable medium 310 provides the same or similarfunctionality as that described with reference to computer-readablemedium 210 of image processing device 200. Second processor 312 providesthe same or similar functionality as that described with reference toprocessor 212 of image processing device 200. Second keyboard 314provides the same or similar functionality as that described withreference to keyboard 214 of image processing device 200. Second mouse316 provides the same or similar functionality as that described withreference to mouse 216 of image processing device 200. Second display320 provides the same or similar functionality as that described withreference to display 218 of image processing device 200. Second speaker322 provides the same or similar functionality as that described withreference to speaker 220 of image processing device 200. Second printer324 provides the same or similar functionality as that described withreference to printer 222 of image processing device 200.

Camera 318 is configured to record and store image data created based onan image received through a lens of camera 318 as understood by a personof skill in the art. Camera 318 further may be accessible by datacollection device 300 through second communication interface 308.

Data collection application 326 performs operations associated withcollecting and storing the image data created by camera 318. Some or allof the operations described herein may be embodied in data collectionapplication 326. The operations may be implemented using hardware,firmware, software, or any combination of these methods. With referenceto the example embodiment of FIG. 3, data collection application 326 isimplemented in software (comprised of computer-readable and/orcomputer-executable instructions) stored in second computer-readablemedium 310 and accessible by second processor 312 for execution of theinstructions that embody the operations of data collection application326. Data collection application 326 may be written using one or moreprogramming languages, assembly languages, scripting languages, etc.

With reference to FIG. 4, a block diagram of data storage system 102 ofdata collection system 106 is shown in accordance with an illustrativeembodiment. Data storage system 102 may include a computer of any formfactor. Data storage system 102 may include a third input interface 404,a third output interface 406, a third communication interface 408, athird computer-readable medium 410, a third processor 412, a database414, and a server application 416. Fewer, different, and additionalcomponents may be incorporated into data storage system 102.

Third input interface 404 provides the same or similar functionality asthat described with reference to input interface 204 of image processingdevice 200. Third output interface 406 provides the same or similarfunctionality as that described with reference to output interface 206of image processing device 200. Third communication interface 408provides the same or similar functionality as that described withreference to communication interface 208 of image processing device 200.Third computer-readable medium 410 provides the same or similarfunctionality as that described with reference to computer-readablemedium 210 of image processing device 200. Third processor 412 providesthe same or similar functionality as that described with reference toprocessor 212 of image processing device 200.

Data storage system 102 includes or can access database 414 eitherthrough a direct connection or through network 108 using thirdcommunication interface 408. Third computer-readable medium 410 mayprovide the electronic storage medium for database 414. Database 414 isa data repository for image creation and processing system 100. Database414 may include a plurality of databases that may be organized intomultiple database tiers to improve data management and access. Database414 may utilize various database technologies and a variety of differentformats as understood by those skilled in the art including a filesystem, a relational database, a system of tables, a structured querylanguage database, etc. Database 414 may be implemented as a singledatabase or as multiple databases stored in different storage locationsdistributed over the Internet or other heterogeneous storageinfrastructures.

Server application 416 performs operations associated with accessingdatabase 414 to store or retrieve image data including image dataassociated with an image mosaic. Some or all of the operations describedherein may be embodied in server application 416. The operations may beimplemented using hardware, firmware, software, or any combination ofthese methods. With reference to the example embodiment of FIG. 4,server application 416 is implemented in software (comprised ofcomputer-readable and/or computer-executable instructions) stored inthird computer-readable medium 410 and accessible by third processor 412for execution of the instructions that embody the operations of serverapplication 416. Server application 416 may be written using one or moreprogramming languages, assembly languages, scripting languages, etc.

Server application 416 may be implemented as a Web application. Forexample, server application 416 may be configured to accept hypertexttransport protocol (HTTP) requests from client devices such as thoseassociated with image processing system 104 and data collection system106 and to send HTTP responses along with optional additional datacontent which may include web pages such as hypertext markup language(HTML) documents and linked objects in response to the HTTP requests.

Data collection application 326, image processing application 224, andserver application 416 may save or store data to database 414 and accessor retrieve data from database 414. Data collection application 326,image processing application 224, and server application 416 may be thesame or different applications or part of an integrated, distributedapplication supporting some or all of the same or additional types offunctionality as described herein. As an example, the functionalityprovided by image processing application 224 may be provided as part ofan integrated image processing application such as Adobe® Photoshop®offered by Adobe Systems Incorporated and/or image processingapplications offered by other software vendors.

In an alternative embodiment, image creation and processing system 100need not include data storage system 102. For example, database 414 maybe stored in computer-readable medium 210 of image processing device 200or second computer-readable medium 310 of data collection device 300. Inanother alternative embodiment, image creation and processing system 100need not include image processing device 200 or data storage system 102.For example, image processing application 224 and server application 416may be integrated and stored in second computer-readable medium 310 ofdata collection device 300. Other levels of integration between thecomponents of image creation and processing system 100 may beimplemented without limitation as understood by a person of skill in theart. For example, all of the functionality described with reference toimage creation and processing system 100 may be implemented in a singlecomputing device.

With reference to FIG. 5, a block diagram illustrating interactionsamong the components of image creation and processing system 100 isshown in accordance with an illustrative embodiment. A first imagecreation and processing system 100 a may include image processing device200, data storage system 102, and data collection device 300, which arein communication using network 108 (not shown). Camera 318 is used tocapture image data that may be stored locally by data collection device300. The image data further may be stored remotely to data storagesystem 102. Image processing device 200 may be used to select all or asubset of the image data stored by data collection device 300 or datastorage system 102 for combining into a image mosaic. Data associatedwith the image mosaic may be stored at data storage system 102 or imageprocessing device 200.

As an example, a first image mosaic 500 is formed by image processingdevice 200 using a first image 502, a second image 504, a third image506, and a fourth image 508 captured by data collection device 300.First image 502, second image 504, third image 506, and fourth image 508are arranged generally to form a 2×2 array of images. A first seam 510and a second seam 512 form the boundary between first image 502 andsecond image 504. A third seam 514 forms the boundary between secondimage 504 and fourth image 508. A fourth seam 516 forms the boundarybetween third image 506 and fourth image 508. A fifth seam 518 forms theboundary between first image 502 and third image 506. A constraint point520 is defined between first seam 510 and second seam 512 by a userinteracting with image processing application 224 of image processingdevice 200. Constraint point 520 may be dragged between first image 502and second image 504 by a user so that the user can visualize the effecton first image mosaic 500. For example, with reference to FIG. 6, asecond image mosaic 600 is formed by image processing device 200 usingfirst image 502, second image 504, third image 506, and fourth image508. Second image mosaic 600 shows constraint point 520 at a secondposition after being dragged and dropped from its first position in FIG.5 to the second position in FIG. 6 as understood by a person of skill inthe art. As a result of the movement of constraint point 520, first seam510 has been moved to include more of second image 504 and acorresponding less of first image 502.

With reference to FIG. 7, example operations associated with imageprocessing application 224 are described. Additional, fewer, ordifferent operations may be performed depending on the embodiment. Forexample, image processing application 224 may provide additionalfunctionality beyond the capability to create an image mosaic andsupport editing of the created image mosaic by a user. As an example,image processing application 224 may reference functionality provided aspart of an integrated image processing application such as those offeredby Adobe Systems Incorporated and/or other software vendors.

The order of presentation of the operations of FIG. 7 is not intended tobe limiting. A user can interact with one or more user interface windowspresented to the user in display 218 under control of image processingapplication 224 independently or through use of a browser application inan order selectable by the user. Thus, although some of the operationalflows are presented in sequence, the various operations may be performedin various repetitions, concurrently, and/or in other orders than thosethat are illustrated. For example, a user may execute image processingapplication 224 which causes presentation of a first user interfacewindow, which may include a plurality of menus and selectors such asdrop down menus, buttons, text boxes, hyperlinks, etc. associated withimage processing application 224 as understood by a person of skill inthe art.

Because of the subjective nature of the image boundaries (seams) withinthe image mosaic and the possibility of various algorithmic techniquesfalling into bad local minima, image processing application 224 supportsa guided interactive technique for creating an image mosaic so that theuser is interjected into the seam boundary problem. Existingalternatives require the manual editing, pixel by pixel, of theindividual image boundaries, which is a time-consuming and tediousprocess where the user relies on perception alone to determine if themanual seam is acceptable. Image processing application 224 allows usersto include or remove dynamic elements, to move an image seam out of apossible local minima into a lower error state, to move the seam into ahigher error state, but one with more acceptable visual coherency, or tohide error in locations where the user determines that the error is lessnoticeable. During the edits, the user is provided the optimal seamsgiven these new constraints.

In an operation 700, a selection of a plurality of image files to readis received. For example, the user may select the files containing thedata for first image 502, second image 504, third image 506, and fourthimage 508 from a drop down list or by browsing to a folder containingimage data files on image processing device 200, data collection device300, and/or data storage device 102. In an illustrative embodiment, theselected plurality of image files are flat, post-registered rasterimages with no geometry except the image offset. Any image data inputcan be converted into this format. Moreover, this generality allows theeasy insertion into any panorama creation pipeline. Example tools usedto form the flat, post-registered raster images include Adobe®Photoshop™ available from Adobe Systems Incorporated, Hugin availablethrough SourceForge.Net®, Kolor Autopano available from Kolor S.A.R.L.,Challes-les-Eaux, France, Gigapan® Stitch.Efx™ available from GigaPanSystems, and PTgui available from New House Internet Services B.V.,Rotterdam, The Netherlands.

After registration, image mosaics contain areas where pixel values areduplicated where individual images overlap. Thus, determining whichpicture provides the color for a pixel location is an important step.The simplest approach is an alpha-blend of the overlap areas to achievea smooth transition between images though other blending techniques areunderstood by a person of skill in the art. For example, a moresophisticated approach is to compute the optimal boundaries betweenimages using a minimization based on the determined energy function(s).

In an illustrative embodiment, seam trees are calculated based purely onpairwise boundary solutions. This technique is fast and highly paralleland shows a significant speed-up compared to previous work even when runsequentially. Some of the seminal work in the formation of image mosaicsassumes that the plurality of images is acquired in a single sweep(either pan, tilt or a combination of the two) of the scene. In suchpanoramas, only pairwise overlaps of images need to be considered. Thepairwise boundaries, which have globally minimal energy, can be computedquickly and exactly using, for example, a min-cut or min-path algorithm.However, these pairwise techniques were thought to not be general enoughto handle the many configurations possible in more complex mosaicconfigurations.

In an operation 702, an energy function to use in computing the imagemosaic from the selected plurality of image files is determined. Thedetermined energy function may be used on the entire image mosaic, on anindividual image formed from a single image file of the plurality ofimage files, or the overlap between images. The energy function may bedefined as a default value that may be changed by the user as understoodby a person of skill in the art. As an example, the user may be providedthe ability to switch between pixel difference and gradient differenceenergies or to provide a custom definition of the energy function used.In an illustrative embodiment, if more than one energy function isdefined, a simple slider provided by the user interface or via inputprovided to the system at launch may be used to blend the energyfunctions applied linearly.

In an operation 704, a number of core processors of image processingdevice 200 is determined. In an illustrative embodiment, the processingof the image files to form the image mosaic may be parallelized based onthe number of core processors. The user may select/enter the number ofcore processors, a default number of core processors may be defined,image processing application 224 may detect the number of coreprocessors, etc. For example, the default number of core processors maybe defined as one, which may result in sequential processing.

In an operation 706, the image file data is read from the selectedplurality of image files. In an operation 708, initial seam trees aredefined based on the image file data using the determined energyfunction(s). The initial intersection computation is computed using therasterized boundaries. Due to aliasing, there may be many intersectionsfound. If the intersections are contiguous, they may be treated as thesame intersection and a representative point chosen. In practice, thischoice has been found to affect the seam in only a very smallneighborhood (less than 10 pixels around the intersection). Therefore,the minimal point in this group may be selected in terms of the energy.Pairs of intersections that are very close in terms of Euclideandistance (i.e., less than 20 pixels) are considered to beinfinitesimally small seams and may be ignored.

Parallel computation may be provided using a thread pool equal to thedetermined number of core processors. Two seam trees are calculated foreach seam. The initial dual seam tree computation for each seam can berun trivially in parallel. In an illustrative embodiment, each seam treeis stored as two buffers: a node buffer, which encodes a node distance,and a tree buffer, which encodes the tree itself. The tree bufferencodes the tree with each pixel referencing its parent. The tree buffercan be encoded as a direction in 2 bits (rounded to 1 byte for speed)for a 4 pixel neighborhood. The node distance (single-source-all-paths),using floating point precision, can be encoded using 4 bytes. Thus, inan illustrative embodiment, 5 bytes per pixel may be used to store theseam tree.

Computing the branching points may also be implemented as a parallelcomputation. In the presence of two adjacent multi-overlaps, theirshared seam is updated in-turn using a mutex flag.

In an operation 710, the image mosaic is presented in display 218 withindicators that show the seam locations. For example, first image mosaic500 is presented to the user with dashed lines indicating the seamsbetween images. In an operation 712, a user interaction with thepresented image mosaic is received. For example, a user movement ofconstraint point 520 may be detected by image processing application 224as understood by a person of skill in the art.

In an operation 714, a determination is made concerning whether or notthe user interaction is to add a constraint to a seam. If thedetermination is that the user interaction is to add a constraint to aseam, in an operation 716, a new seam tree is defined as discussedfurther below and the image mosaic presented in display 218 is updatedto reflect the new constraint and resulting new seam location, andprocessing continues in an operation 718. If the determination is thatthe user interaction is not adding a constraint to a seam, processingcontinues in operation 718.

In an operation 718, a determination is made concerning whether or notthe user interaction is to move a constraint in a seam. If thedetermination is that the user interaction is to move a constraint in aseam, in an operation 720, the seam tree defined for the constraint isupdated as discussed further below and the image mosaic presented indisplay 218 is updated to reflect the new constraint position andresulting new seam location, and processing continues in an operation722. If the determination is that the user interaction not moving aconstraint in a seam, processing continues in operation 722.

The adding of a constraint to a seam and its movement may be referred toas a bending interaction. With reference to FIG. 8 a, the boundary linesof a first image 800 and a second image 802 intersect at a first point804 denoted u and at a second point 806 denoted v, which are connectedby a seam 808 denoted s. In a more general case, with reference to FIG.8 b, the boundary lines of a third image 810 and a fourth image 812result in an even number of intersection points. A set of seams can bebuilt by connecting pairs of points with a consistent winding such as u₁to v₁, u₂ to v₂, u₃ to v₃, and u₄ to v₄, or vice versa. Given a simpleoverlap configuration, a seam can be thought of as a path that connectspairs of boundary intersections u and v. Seams computed in this waydefine a complete partition of the image mosaic space between thirdimage 810 and fourth image 812. In non-simple cases, for example, withco-linear boundary intervals, the same result can be achieved bychoosing one representative point that may be optimized to minimize someenergy. Each seam u₁ to v₁, u₂ to v₂, u₃ to v₃, and u₄ to v₄ can betreated as independent.

A user constraint can be considered a point or a region marked in such away that the seam either does or does not pass through it. The region ofinclusion or exclusion may have an arbitrary size denoted by the user oran external program. To find a minimal seam which does pass though adefined region, a pixel location with minimal tree distance from theproper seam tree endpoints can be determined. Constraining a seam topass through the defined region may be determined by applying weights insuch a way that the optimal seam passes through the defined regionduring calculation. An example of this weighting is to give the definedregion a small energy compared to the rest of the pairwise overlap areaof the images forming the image mosaic. To find a minimal seam whichdoes not pass though a defined region, a pixel location with minimaltree distance from the proper seam tree endpoints can be determined.Constraining a seam to not pass through the defined region may bedetermined by applying weights in such a way that the optimal seam doesnot pass through the defined region during calculation. An example ofthis weighting is to give the defined region a very large or infiniteenergy compared to the rest of the pairwise overlap area of the imagesforming the image mosaic.

Given a collection of n panorama images I₁, I₂, . . . , I_(n) and thepanorama P, the image boundary problem can be thought of as finding adiscrete labeling L(p)∈(1 . . . n) for all panorama pixels p∈P whichminimizes the transition between each image. If L(p)=k, this indicatesthat the pixel value for location p in the panorama comes from imageI_(k). The transition can be defined by an energy on the piecewisesmoothness E_(s)(p,q) of the labeling of neighboring elements p, q∈

, where

r is the set of all neighboring pixels. Typically, the sum of the energyof all neighbors, E, is minimized. For the panorama boundary problem,this energy can be defined as:

E(L)=

E _(s)(p,q).

If minimizing the transition in pixel values:

E _(s)(p,q)=

|I _(L(p))(p)−I _(L(q))(p)|

+

|I _(L(p))(q)−I _(L(q))(q)|

or if minimizing the transition in the gradient:E _(s)(p,q)=

|∇I _(L(p))(p)−∇I _(L(q))(p)|

+

|∇I _(L(p))(q)−∇I _(L(q))(q)|

where L(p) and L(q) are the labeling of the two pixels. Notice thatL(p)=L(q) implies that E_(s)(p,q)=0. Minimizing the change in pixelvalue works well in the context of poor registration or moving objectsin the scene, while minimizing the gradient produces a nice input fortechniques such as gradient domain blending. In addition, techniques mayuse a linear combination of the two smoothness energies.

When computing the optimal boundary between two images, the binarylabeling may be computed using a min-cut of a graph whose nodes are thepixels and whose arcs connect a pixel to its neighbors. The arc weightsare the energy function being minimized. For example, a 4-neighborhoodmin-cut solution is shown with reference to FIG. 9 a with its dualmin-path solution shown with reference to FIG. 9 b. The min-cut labelingis shown using gradient fill with a first plurality of pixels 800indicating the first image and a second plurality of pixels 802indicating the second image. A min-path 804 is shown in FIG. 9 b. Thus,there is an equivalent min-path 804 to the min-cut solution shown inFIG. 9 a. This has been shown to be true for all single source, singledestination paths. Min-path 804 can be computed with a min-pathalgorithm such as Dijkstra's algorithm described in Dijkstra, E. W. Anote on two problems in connexion with graphs, Numerische Mathematik 1,269-271 (1959). Dijkstra's algorithm solves the single-source shortestpath problem for a graph with nonnegative edge path costs and produces ashortest path tree. The approaches are equivalent in the sense that thefinal solution of a min-cut calculation defines the pixel labeling asshown in FIG. 9 a while the min-path solution defines the path thatseparates pixels of different labeling as shown in FIG. 9 b.

Assuming the dual-path energy representation as shown in FIG. 9 b, withreference to FIG. 10, a seam 1000 is a path that connects a firstintersection point 1002 to a second intersection point 1004. Computingthe minimal path using the determined energy function(s) gives seam1000, which can be computed efficiently with Dijkstra's algorithm. Withminimal additional overhead, a first min-path tree 1006 and a secondmin-path tree 1008 can be computed from first intersection point 1002and second intersection point 1004 (single source all paths). Firstmin-path tree 1006 and second min-path tree 1008 provide all minimalseams which originate from either first intersection point 1002 orsecond intersection point 1004 and define a dual seam tree. As discussedpreviously, first min-path tree 1006 and second min-path tree 1008 maybe stored as two buffers: a node buffer, which encodes the nodedistance, and a tree buffer, which encodes the tree itself.

Given a point 1010 in the image overlap, a first minimal path 1012 tofirst intersection point 1002 and a second minimal path 1014 to secondintersection point 1004 can be found with a linear walk up of firstmin-path tree 1006 and second min-path tree 1008, respectively. Thus,given two min-path trees associated with seam endpoints, a new seam thatpasses through any point in the overlap region can be determined using asimple linear walk up each tree.

If this point is a user constraint, the union of the two minimal pathsforms a new constrained optimal seam. Due to the simplicity of thelookup, the new path computation is fast enough to achieve interactiverates even for large image overlaps. Two min-paths on the same energyfunction are guaranteed not to cross. Given that each dual-seam tree iscomputed independently though, the paths can cross if there are multiplemin-path solutions. This may happen if there are multiple seams whichhave the same exact energy and the tree computed by Dijkstra's algorithmis dependent on the order in which the edges are calculated. To avoidthis problem, an ordering is enforced based on an edge index to achievenon-crossing solutions.

Moving a constraint (or endpoint) is also a simple walk up its partnerendpoint's (or constraint's) seam tree. Therefore, a user can change aconstraint or endpoint location at-will, interactively. After themovement, the shifted endpoint's (or constraint's) seam tree is nolonger valid because it is based on a previous location. If futureinteractions are desired, the seam tree is updated based on the movedconstraint or endpoint location in operation 718. The seam tree updatecan be computed as a background process after the user finishes theinteraction without any loss of responsiveness by image processingapplication 224.

Adding a constraint is akin to splitting the seam into segments.Splitting may occur, for example, when a user releases mouse 216 after abending interaction. After a bending interaction, the seam is split intoseparate seam structures. Now four seam trees exist to describe the seaminstead of the previous two seam trees. Two of the trees (i.e., firstmin-path tree 1006 and second min-path tree 1008 corresponding to theendpoints) are inherited by the new seams. The two new trees associatedwith the new constraint are identical; therefore, only one additionaltree computation is needed. The new seam tree also may be stored as anode buffer and a tree buffer. The new seam tree provides all minimalseams which originate from the constraint location. Editing may belocked for the seam until the seam trees are updated to include the newseam tree.

Thus, after a user interaction adding a constraint to a seam, inoperation 716, a new seam tree is defined and the updated image mosaicpresented in display 218. After a user interaction moving a constraintin a seam, in operation 720, the seam tree associated with theconstraint is updated and the updated image mosaic presented in display218. Thus, image processing application 224 supports a user's additionof a constraint to a seam, a user's movement of a constraint, and theoptimal seam which must pass through this new or moved constraint iscomputed almost instantly. The constraint can be moved interactively bythe user to explore the seam solution space.

Pairwise seams can be recombined by computing seam intersections, whichmay be called branching points in overlap clusters. Specifically,clusters are groups of overlaps that share a common area called amulti-overlap. For example, if at least three of the overlapping imagesoverlap at one or more pixels in the image mosaic, a multi-overlap areacan be defined. Of course, more than three of the images may overlap insome multi-overlap areas. A determination as to which overlap areas areneeded to compute a valid global seam network can be provided by userinput or by an external algorithm. For example, for panoramas taken witha robotic tripod head the images lie on a grid. A configuration of validoverlaps given this regular structure can be the vertical and horizontaloverlaps between adjacent images on the grid.

Assuming that for each pairwise seam there exist only two endpoints, foreach multi-overlap one endpoint is adapted into a branching point. Thisendpoint may be referred to as being inside in relation to themulti-overlap. The other seam endpoint may be referred to as beingoutside in relation to the multi-overlap. A branching point can becomputed by finding the endpoints that are closest, for example, basedon a Euclidean distance to the multi-overlap area. Although it ispossible to create a pathological overlap configuration where thisdistance metric fails, this strategy has been shown to work well inpractice.

As an example, with reference to FIG. 11, a first image 1100, a secondimage 1102, a third image 1104, and a fourth image 1106 are shownincluding a multi-overlap area 1108. A first set of intersectionendpoints between the images 1100, 1102, 1104, 1106 is defined formulti-overlap area 1108. The first set of intersection endpointsincludes a first overlap endpoint 1110 between first image 1100 andsecond image 1102; a second overlap endpoint 1112 between first image1100 and third image 1104; a third overlap endpoint 1114 between secondimage 1102 and fourth image 1106; and a fourth overlap endpoint 1116between third image 1104 and fourth image 1106. The overlap endpointsare referred to as being inside in relation to multi-overlap area 1108.A second set of intersection endpoints between the images 1100, 1102,1104, 1106 is defined outside of multi-overlap area 1108. The second setof intersection endpoints includes a first endpoint 1111 between firstimage 1100 and second image 1102; a second endpoint 1113 between firstimage 1100 and third image 1104; a third endpoint 1115 between secondimage 1102 and fourth image 1106; and a fourth endpoint 1117 betweenthird image 1104 and fourth image 1106. These endpoints are referred toas being outside in relation to multi-overlap area 1108.

The first set of intersection endpoints is adapted into a branchingpoint. The branching point can be determined by finding a minimum pointin the multi-overlap with respect to min-path distance from the partnerendpoints (the second set of intersection endpoints, respectively). Forexample, with reference to FIG. 12, a branching point 1200 is calculatedby finding a minimum point in multi-overlap area 1108 with respect tomin-path distance from first overlap endpoint 1110 to first endpoint1111, from second overlap endpoint 1112 to second endpoint 1113, fromthird overlap endpoint 1114 to third endpoint 1115, and from fourthoverlap endpoint 1116 to fourth endpoint 1117. If dual seam treedistances are used, i.e. the path distance values associated with thesecond set of intersection endpoints, branching point 1200 can becomputed that is optimal with respect to the four paths. For example,the distance values can be obtained by a simple lookup in the overlappoint node buffers discussed previously. Branching point 1200 may bedetermined by minimizing a sum of a squared distance for each seam(i.e., first overlap endpoint 1110 to first endpoint 1111, secondoverlap endpoint 1112 to second endpoint 1113, third overlap endpoint1114 to third endpoint 1115, and fourth overlap endpoint 1116 to fourthendpoint 1117).

After branching point 1200 is found, the new seams may be computed by alinear lookup up the partner endpoint's (i.e., first endpoint 1111,second endpoint 1113, third endpoint 1115, and fourth endpoint 1117)corresponding seam tree as shown with reference to FIG. 13 and describedpreviously with reference to FIG. 10. Thus, the seam tree associatedwith the adjusted endpoint is recalculated given the new root locationdefined by branching point 1200.

In practice, the contribution of the root location has been found tohave a minimal affect on the overall structure of the seam tree as theleaves of the seam tree are approached. To enable parallel computation,each branching point may be computed using an initial endpoint locationeven if the endpoint is moved via another branching point calculation.For example, as shown with reference to FIG. 14, fourth overlap endpoint1116 is used to compute a branching point for a second multi-overlaparea 1400 between third image 1104, fourth image 1106, a fifth image1402, and a sixth image 1404, instead of branching point 1200. Thus,since using the initial starting endpoints allows the branching pointsto be computed independently and in a single parallel pass, redefiningthe seam trees after movement of a branching point may apply thisapproach. The seams may be combined sequentially in the same manner asthe technique described in A. A. Efros, & W. T. Freeman, Image quiltingfor texture synthesis and transfer, SIGGRAPH, 341-346 (2001).

Thus, image processing application 224 supports a user's ability to graband move a branching point associated with a selected multi-overlaparea. As discussed previously, a movement of an endpoint is a simplelookup on its partner's dual seam tree. As the user moves a branchingpoint, intersections for both the selected multi-overlap and alladjacent multi-overlaps are resolved. After movement, to enable furtherinteraction, the seam trees associated with the moved branching pointare redefined in operation 724 and the updated image mosaic presented indisplay 218. When the user releases mouse 216, the seam tree data forall the points associated with the active seams for the multi-overlapare recomputed as a background process in parallel. Like splitting,editing may be locked for each seam until the seam tree update iscompleted.

In an operation 726, a determination is made concerning whether or notanother user interaction is received. If the determination is thatanother user interaction is received, processing continues in operation714. If the determination is that another user interaction is notreceived, processing continues in operation 726 to wait for the nextuser interaction. Of course, other interactions that may be supported byimage processing application 224 are the ability to zoom, pan, tilt,color correction, etc.

With reference to FIG. 15, a portion of a first user interface window1500 is shown in accordance with an example embodiment after the userexecutes image processing application 224. First user interface window1500 presents a first image mosaic 1501 composed of nine images.Relative to the nine images there were moving objects during acquisitionof the images, registration issues between the images, and a varyingexposure between the images. First user interface window 1500 furtherincludes a movement category button 1502, a branching point movementbutton 1504, a user constraint movement button 1506, a user constraintrepetition button 1508, an add/remove category button 1510, and aninput/output (I/O) category button 1512.

Movement category button 1502 represents a movement category of buttons,add/remove category button 1510 represents an add/remove category ofbuttons, and I/O category button 1512 represents an I/O category ofbuttons. Movement category button 1502 acts as a folder that containsbranching point movement button 1504, user constraint movement button1506, and user constraint repetition button 1508. Thus, selection ofmovement category button 1502 causes branching point movement button1504, user constraint movement button 1506, and user constraintrepetition button 1508 to be displayed and a subsequent selection ofmovement category button 1502 causes branching point movement button1504, user constraint movement button 1506, and user constraintrepetition button 1508 to not be displayed. Selection of branching pointmovement button 1504 allows the user to select and drag a branchingpoint included in one or more seams of first image mosaic 1501.Selection of user constraint movement button 1506 allows the user toselect and drag a constraint point included in one or more seams offirst image mosaic 1501. Selection of user constraint repetition button1508 allows the user to repeatedly add user constraint points usingmouse 216. The resulting new seam may be presented to the userautomatically in display 218 based on a position of mouse 216 andcalculation of a “seam split” when the user clicks mouse 216 asdiscussed previously.

With reference to FIG. 16, a portion of a second user interface window1600 is shown in accordance with an example embodiment after the usercontinues to interact with first user interface window 1500. Second userinterface window 1600 presents the nine images included in first imagemosaic 1501 as a second image mosaic 1601 after further interaction bythe user using one or more of the controls of movement category button1502 and/or add/remove category button 1510. Second user interfacewindow 1600 includes an add user constraint button 1602 and a removeuser constraint button 1604. Add/remove category button 1510 acts as afolder that contains add user constraint button 1602 and remove userconstraint button 1604. Thus, selection of add/remove category button1510 causes add user constraint button 1602 and remove user constraintbutton 1604 to be displayed and a subsequent selection of add/removecategory button 1510 causes add user constraint button 1602 and removeuser constraint button 1604 to not be displayed. Selection of add userconstraint button 1602 allows the user to add a constraint point tosecond image mosaic 1601. Selection of remove user constraint button1604 allows the user to remove a constraint point from second imagemosaic 1601. As discussed previously, adding a constraint causescreation of a new seam tree associated with the new constraint locationand deletion of a constraint causes deletion of the associated new seamtree.

With reference to FIG. 17, a portion of a third user interface window1700 is shown in accordance with an example embodiment after the usercontinues to interact with first user interface window 1500. Third userinterface window 1700 presents the nine images included in first imagemosaic 1501 as a third image mosaic 1701 that has been zoomed to showadditional details of a portion of the nine images after furtherinteraction by the user using one or more of the controls of movementcategory button 1502 and/or add/remove category button 1510. Third userinterface window 1700 includes a load seams button 1702 and a save seamsbutton 1704. I/O category button 1512 acts as a folder that containsload seams button 1702 and save seams button 1704. Thus, selection ofI/O category button 1512 causes load seams button 1702 and save seamsbutton 1704 to be displayed and a subsequent selection of I/O categorybutton 1512 causes load seams button 1702 and save seams button 1704 tonot be displayed. Selection of load seams button 1702 allows the user toload seams for third image mosaic 1701. Thus, in an illustrativeembodiment, the seams can be saved and reloaded for future editing. Theseams can be saved as a labeling of pixels, where the color valuedenotes the image that provides the pixel in the image mosaic. Anadditional file may store the mapping of colors to image files.Additionally or in the alternative, each seam's geometry may be directlysaved as integer (x, y) coordinates. Selection of save seams button 1704triggers saving of the currently calculated seams for third image mosaic1701.

The word “illustrative” is used herein to mean serving as an example,instance, or illustration. Any aspect or design described herein as“illustrative” is not necessarily to be construed as preferred oradvantageous over other aspects or designs. Further, for the purposes ofthis disclosure and unless otherwise specified, “a” or “an” means “oneor more”. Still further, the use of “and” or “or” is intended to include“and/or” unless specifically indicated otherwise. The illustrativeembodiments may be implemented as a method, apparatus, or article ofmanufacture using standard programming and/or engineering techniques toproduce software, firmware, hardware, or any combination thereof tocontrol a computer to implement the disclosed embodiments.

The foregoing description of illustrative embodiments of the inventionhas been presented for purposes of illustration and of description. Itis not intended to be exhaustive or to limit the invention to theprecise form disclosed, and modifications and variations are possible inlight of the above teachings or may be acquired from practice of theinvention. The embodiments were chosen and described in order to explainthe principles of the invention and as practical applications of theinvention to enable one skilled in the art to utilize the invention invarious embodiments and with various modifications as suited to theparticular use contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto and theirequivalents.

What is claimed is:
 1. A computer-readable medium having stored thereoncomputer-readable instructions that when executed by a computing devicecause the computing device to: control presentation of an image mosaicin a display of a first device, wherein the image mosaic is created froma plurality of overlapping images, wherein a first seam is shown betweena first pair of the plurality of overlapping images, wherein a firstseam includes a plurality of pixels in the image mosaic, wherein on afirst side of the first seam a first image of the first pair is shownand on a second side of the first seam opposite the first side a secondimage of the first pair is shown; receive an indicator of an interactionby a user with the image mosaic presented in the display, wherein theindicator indicates that the user has selected a pixel of the pluralityof pixels of the first seam and has moved the selected pixel to a firstlocation within an overlapping region between the first pair ofoverlapping images; compute a second seam that includes the firstlocation; and control presentation of the image mosaic in the display ofthe first device, wherein the image mosaic includes the second seambetween the first pair of overlapping images, and further wherein thesecond seam replaces at least a portion of the first seam in the imagemosaic.
 2. The computer-readable medium of claim 1, wherein locations ofthe plurality of pixels that form the first seam are determined based ona first independent pair-wise boundary computation between a firstoverlap point and a second overlap point of the first pair ofoverlapping images.
 3. The computer-readable medium of claim 2, whereinlocations of the plurality of pixels that form the first seam aredetermined based on a second independent pair-wise boundary computationbetween a third overlap point and a fourth overlap point of the firstpair of overlapping images.
 4. The computer-readable medium of claim 3,wherein the locations of the plurality of pixels based on the firstindependent pair-wise boundary computation and on the second independentpair-wise boundary computation are determined using a parallel process.5. The computer-readable medium of claim 2, wherein a number of theplurality of overlapping images is at least three and at least three ofthe plurality of overlapping images overlap at one or more pixels in theimage mosaic.
 6. The computer-readable medium of claim 5, wherein thefirst overlap point is calculated as a minimum sum of a squared distancebetween a plurality of overlap points for a multi-overlapping regionincluding the one or more pixels.
 7. The computer-readable medium ofclaim 2, wherein the second seam is computed using a first seam tree anda second seam tree, wherein the first seam tree is stored as a firstnode buffer and a first tree buffer, wherein the first node bufferincludes a first node distance value for each pixel in the overlappingregion to the first overlap point and the first tree buffer includes afirst direction value for each pixel in the overlapping region thatindicates a first direction of a first connecting pixel in a 4-pixelneighborhood about each pixel in a first path from the first overlappoint to the respective pixel, and further wherein the second seam treeis stored as a second node buffer and a second tree buffer, wherein thesecond node buffer includes a second node distance value for each pixelin the overlapping region to the second overlap point and the secondtree buffer includes a second direction value for each pixel in theoverlapping region that indicates a second direction of a secondconnecting pixel in a 4-pixel neighborhood about each pixel in a secondpath from the second overlap point to the respective pixel.
 8. Thecomputer-readable medium of claim 7, wherein the first path from thefirst overlap point is determined using a min-path algorithm.
 9. Thecomputer-readable medium of claim 8, wherein the second path from thesecond overlap point is determined using the min-path algorithm.
 10. Thecomputer-readable medium of claim 9, wherein the second seam is computedby identifying a first pixel in the first node buffer associated withthe first location and by walking up the first seam tree to the firstoverlap point from the first pixel using the first node distance valueand the first direction value.
 11. The computer-readable medium of claim10, wherein the second seam is further computed by identifying a secondpixel in the second node buffer associated with the first location andby walking up the second seam tree to the second overlap point from thesecond pixel using the second node distance value and the seconddirection value.
 12. The computer-readable medium of claim 8, whereinthe min-path algorithm uses an energy function to minimize a transitionbetween the first pair of overlapping images.
 13. The computer-readablemedium of claim 12, wherein the energy function minimizes a pixeltransition or a gradient transition between the first pair ofoverlapping images.
 14. The computer-readable medium of claim 1, whereinthe first location is defined by a plurality of points which define aregion and the second seam includes at least a portion of the region.15. The computer-readable medium of claim 7, wherein thecomputer-readable instructions further cause the computing device to:receive a second indicator of a second interaction by the user with theimage mosaic presented in the display, wherein the second indicatorindicates that the user has selected a second pixel of the plurality ofpixels of the second seam and has moved the selected second pixel to asecond location within the overlapping region between the first pair ofoverlapping images; compute a third seam that includes the secondlocation; and control presentation of the image mosaic in the display ofthe first device, wherein the image mosaic includes the third seambetween the first pair of overlapping images, and further wherein thethird seam replaces at least a portion of the second seam in the imagemosaic.
 16. The computer-readable medium of claim 15, wherein the thirdseam is computed using a third seam tree, wherein the third seam tree isstored as a third node buffer and a third tree buffer, wherein the thirdnode buffer includes a third node distance value for each pixel in theoverlapping region to the first location and the third tree bufferincludes a third direction value for each pixel in the overlappingregion that indicates a third direction of a third connecting pixel in a4-pixel neighborhood about each pixel in a third path from the firstlocation to the respective pixel.
 17. The computer-readable medium ofclaim 15, wherein the computer-readable instructions further cause thecomputing device to: receive a third indicator of a third interaction bythe user with the image mosaic presented in the display, wherein thethird indicator indicates that the user has deleted the pixel of theplurality of pixels of the first seam; compute a fourth seam thatremoves the pixel from the third seam; and control presentation of theimage mosaic in the display of the first device, wherein the imagemosaic includes the fourth seam between the first pair of overlappingimages, and further wherein the fourth seam replaces at least a portionof the third seam in the image mosaic.
 18. The computer-readable mediumof claim 1, wherein the computer-readable instructions further cause thecomputing device to: receive a second indicator of a second interactionby the user with the image mosaic presented in the display, wherein thesecond indicator indicates that the user has selected a second pixel ofthe plurality of pixels of the second seam and has indicated that thesecond pixel be excluded from the second seam; compute a third seam thatexcludes the second pixel; and control presentation of the image mosaicin the display of the first device, wherein the image mosaic includesthe third seam between the first pair of overlapping images, and furtherwherein the third seam replaces at least a portion of the second seam inthe image mosaic.
 19. A system comprising: a display; a processor; and acomputer-readable medium operably coupled to the processor, thecomputer-readable medium having computer-readable instructions storedthereon that, when executed by the processor, cause the system tocontrol presentation of an image mosaic in the display, wherein theimage mosaic is created from a plurality of overlapping images, whereina first seam is shown between a first pair of the plurality ofoverlapping images, wherein a first seam includes a plurality of pixelsin the image mosaic, wherein on a first side of the first seam a firstimage of the first pair is shown and on a second side of the first seamopposite the first side a second image of the first pair is shown;receive an indicator of an interaction by a user with the image mosaicpresented in the display, wherein the indicator indicates that the userhas selected a pixel of the plurality of pixels of the first seam andhas moved the selected pixel to a first location within an overlappingregion between the first pair of overlapping images; compute a secondseam that includes the first location; and control presentation of theimage mosaic in the display, wherein the image mosaic includes thesecond seam between the first pair of overlapping images, and furtherwherein the second seam replaces at least a portion of the first seam inthe image mosaic.
 20. A method of allowing a user to modify an imagemosaic, the method comprising: presenting an image mosaic in a displayof a first device under control of a processor, wherein the image mosaicis created from a plurality of overlapping images, wherein a first seamis shown between a first pair of the plurality of overlapping images,wherein a first seam includes a plurality of pixels in the image mosaic,wherein on a first side of the first seam a first image of the firstpair is shown and on a second side of the first seam opposite the firstside a second image of the first pair is shown; receiving, by theprocessor, an indicator of an interaction by a user with the imagemosaic presented in the display, wherein the indicator indicates thatthe user has selected a pixel of the plurality of pixels of the firstseam and has moved the selected pixel to a first location within anoverlapping region between the first pair of overlapping images;computing, by the processor, a second seam that includes the firstlocation; and presenting the image mosaic in the display of the firstdevice under control of the processor, wherein the image mosaic includesthe second seam between the first pair of the plurality of overlappingimages, wherein the second seam replaces at least a portion of the firstseam in the image mosaic.